Search for: All records

We present ActSonic, an intelligent, low-power active acoustic sensing system integrated into eyeglasses that can recognize 27 different everyday activities (e.g., eating, drinking, toothbrushing) from inaudible acoustic waves around the body. It requires only a pair of miniature speakers and microphones mounted on each hinge of the eyeglasses to emit ultrasonic waves, creating an acoustic aura around the body. The acoustic signals are reflected based on the position and motion of various body parts, captured by the microphones, and analyzed by a customized self-supervised deep learning framework to infer the performed activities on a remote device such as a mobile phone or cloud server. ActSonic was evaluated in user studies with 19 participants across 19 households to track its efficacy in everyday activity recognition. Without requiring any training data from new users (leave-one-participant-out evaluation), ActSonic detected 27 activities, achieving an average F1-score of 86.6% in fully unconstrained scenarios and 93.4% in prompted settings at participants' homes. 

Sensing movements and gestures inside the oral cavity has been a long-standing challenge for the wearable research community. This paper introduces EchoNose, a novel nose interface that explores a unique sensing approach to recognize gestures related to mouth, breathing, and tongue by analyzing the acoustic signal reflections inside the nasal and oral cavities. The interface incorporates a speaker and a microphone placed at the nostrils, emitting inaudible acoustic signals and capturing the corresponding reflections. These received signals were processed using a customized data processing and machine learning pipeline, enabling the distinction of 16 gestures involving speech, tongue, and breathing. A user study with 10 participants demonstrates that EchoNose achieves an average accuracy of 93.7% in recognizing these 16 gestures. Based on these promising results, we discuss the potential opportunities and challenges associated with applying this innovative nose interface in various future applications. 

We propose the use of interactive vignettes as an alternative to traditional text- and video-based vignettes for conducting large-scale Human-Robot Interaction (HRI) studies. Interactive vignettes maintain the advantages of traditional vignettes while offering additional affordances for participant interaction and data collection through interactive elements. We discuss the core affordances of interactive vignettes, including explorability, responsiveness, and non-linearity, and look into how these affordances can enable HRI research with more complex scenarios. To demonstrate the strength of the approach, we present a case study of our own research project with N=87 participants and show the data we collect through interactive vignettes. We suggest that the use of interactive vignettes can benefit HRI researchers in learning how participants interact with, respond to, and perceive a robot’s behavior in pre-defined scenarios.